1. Field of the Invention
The present invention relates to imaging systems, and more particularly to a system for aligning images obtained from multiple imaging devices.
2. Description of the Related Art
For many military, civilian, or industrial applications, such as industrial machine vision, military imaging, survey operations, etc., multiple images may be combined into one large and continuous image (e.g., a panoramic image). The process of combining “off-line” multiple images into one large and continuous image is typically computationally intensive. The “off-line” image combination process starts with correcting geometrical distortions associated with the individual images. The process continues with the identification and/or generation of image references within each individual image. The image references may be fixed references, such as, for example, edges of images. The image references may also be selected as easily identifiable reference objects, such as pre-positioned or projected markers. After image references for individual images have been identified or generated, the references of different images are aligned, and maps for an overall partition of the panoramic image are compiled. The partition separates non-overlapping images from overlapping images, and assigns different algorithms to non-overlapping and overlapping images. With one algorithm, overlapping images are warped and matched along seams. The output pixels for the group of overlapping images are generated by interpolating source pixels in the images. With another algorithm, non-overlapping images are warped and attribute-matched to their neighboring images.
The above technique for combining “off-line” multiple images into one image could also be applied to live video images, however, the resulting computational complexity may make such image processing systems bulky and prohibitively expensive. Moreover, the use of multiple buffers for image processing, as may be used for “off-line” image combinations, may bring image latency to levels unacceptable for many applications that use live imaging. The alignment of live video images from multiple cameras, prior to panoramic image stitching, may demand more resources than are available to most systems.
One system for alignment of images for stitching is described in US Patent Application 2003/0142882 A1 by Gabriel Beged-Dov et al. The system described in this patent application generates fiducial marks on adjacent subject scenes, at user-determined locations. The subject scenes are then combined into a stitched image, by matching their fiducial marks. This system, however, requires steps of repositioning image-capturing devices, projecting images after they are captured, and removing fiducial artifacts from the final stitched image.
Disclosed embodiments of this application address these and other issues by providing each imaging device with a secondary optical path for generating a reference image, which is used to determine positioning.